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PREFACE 


This  Report  is  one  of  a  continuing  series  of  studies 
documenting  research  on  group  judgment.  Studies  to  date 
have  indicated  that  substantial  improvements  can  be  made 
in  procedures  for  formulating  group  judgments  in  those 
areas  where  the  best  information  available  is  expert 
opinion. 

Cross-impact  analysis  is  a  method  of  taking  account 
of  the  interrelationships  of  a  set  of  events  in  assessing 
the  individual  probabilities  of  those  events.  It  has  been 
applied  in  studies  of  future  technological  developments 
and  social  trends. 

This  Report  provides  a  logical  foundation  for  an  ele¬ 
mentary  form  of  cross-impact  analysis.  It  should  be  of 
interest  to  military  and  industrial  agencies  concerned 
with  long-range  forecasting  of  technological  and  social 
events.  It  probably  also  has  applications  in  the  study 
of  large  systems  of  events  that  interact  probabilistically. 

The  research  was  supported  by  the  Advanced  Research 
Projects  Agency  of  the  Department  of  Defense. 

Readers  interested  in  other  studies  of  the  group 
judgment  technology  project  should  see  References  3 
through  9. 
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SUMMARY 


Cross-impact  analysis  is  a  method  for  revising  esti¬ 
mated  probabilities  of  future  events  in  terms  of  estimated 
interactions  among  those  events.  This  Report  presents  an 
elementary  cross-impact  model  where  the  cross-impacts  are 
formulated  as  relative  probabilities.  Conditions  are 
derived  for  the  consistency  of  the  matrix  of  relative 
probabilities  of  n  events.  An  extension  also  provides 
a  necessary  condition  for  the  vector  of  absolute  probabil¬ 
ities  to  be  consistent  with  the  relative  probability 
matrix.  An  averaging  technique  is  formulated  for  resolving 
inconsistencies  in  the  matrix,  and  a  nearest-point  computa¬ 
tion  derived  for  resolving  inconsistencies  between  the  set 
of  absolute  probabilities  and  the  matrix. 

Although  elementary,  the  present  model  clarifies  some 
of  the  conceptual  problems  associated  with  cross-impact 
analysis,  and  supplies  a  relatively  sound  basis  for  revis¬ 
ing  probability  estimates  in  the  limited  case  where  inter¬ 
actions  can  be  approximated  by  relative  probabilities. 
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AN  ELEMENTARY  CROSS- IMPACT  MODEL 


N.  C.  Dalkey 


I.  INTRODUCTION 

One  of  the  more  promising  new  tools  for  long-range 
forecasting  is  cross-impact  analysis.  The  general  notion 
was  first  suggested  by  Helmer  and  Gordon  with  the  game 
"Futures"  created  for  the  Kaiser  Corporation.  Cross-impact 
analysis  has  now  been  expanded  and  applied  to  a  number  of 
forecasting  areas  by  Gordon  and  others  at  The  Institute 
for  the  Future  [1],  The  motivation  for  cross-impact  anal¬ 
ysis  arises  from  a  basic  aspect  of  long-range  forecasting. 
There  are  usually  strong  interactions  among  a  set  of  poten¬ 
tial  technological  events  or  among  a  set  of  potential  social 
developments.  In  assessing  the  likelihood  that  any  given 
event  or  development  will  occur,  the  interactions  with  other 
events  are  clearly  relevant.  However,  the  number  of  first- 
order  potential  interactions  increases  as  the  square  of  the 
number  of  events.  Even  if  a  matrix  describing  the  inter¬ 
actions  is  available — say  from  estimates  furnished  by  a 
panel  of  experts--the  task  of  thinking  through  the  implica¬ 
tions  rapidly  gets  out  of  hand.  Some  computational  aid  is 
required  to  take  account  of  the  large  number  of  interde¬ 
pendencies. 

I 
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Gordon  and  others  at  The  Institute  for  the  Future 

have  developed  two  major  approaches  to  the  computational 
* 

program.  Both  approaches  involve  (1)  preliminary  esti¬ 
mates  of  the  absolute  probabilities,  (i.e.,  the  probabili¬ 
ties  of  the  individual  events,  (2)  estimates  of  the  inter¬ 
dependencies  in  terms  of  a  cross- impact  matrix,  (3)  a 
Monte  Carlo  sampling  of  chains  of  events  in  which  the 
probability  of  an  event  in  the  chain  is  modified  by  the 
cross-impact  of  the  previously  occurring  event  in  the 
chain,  and  (4)  reestimation  of  the  absolute  probability 
of  each  event  in  terms  of  the  relative  frequency  of  the 
occurrence  of  that  event  in  the  sample  of  chains.  The 
difference  between  the  two  approaches  lies  in  the  mode 
of  modification  of  the  probabilities.  In  the  first 
approach,  the  basic  method,  the  modification  is  effected 
by  a  heuristic  algorithm.  Cross -impacts  are  rated  on  a 
nominal  scale  of  -10  to  +10.  Modification  of  successive 
probabilities  is  computed  via  a  family  of  quadratic  "ad¬ 
justments,"  based  on  the  cross-impact  rating  and  the 
unmodified  probability.  The  second  approach,  the 

They  actually  have  worked  with  four  variations  of  the 
cross-impact  technique.  Two  of  these  variations,  the  dynam¬ 
ic  model  and  the  scenario  model,  involve  aspects  of  the 
interdependency  problem  (namely,  strict  time  or  order  rela¬ 
tionships)  which  are  beyond  the  scope  of  the  present  paper. 
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likelihood  ratio  method,  defines  cross-impacts  in  terms  of 
a  factor  by  which  the  odds  favoring  the  target  event  are 
to  be  multiplied,  given  the  occurrence  of  the  impacting 
event.  The  second  approach  is  conceptually  clearer  than 
the  first,  and  removes  some  of  the  arbitrariness  associated 
with  it. 

Both  approaches  suffer  from  a  lack  of  clarity  concern¬ 
ing  the  purpose  of  the  computation.  The  notion  of  "taking 
account  of  the  interactions"  is  not  adequate  to  answer 
questions  such  as  "are  the  revised  probabilities  in  fact 
more  accurate  estimates  than  the  original  ones"?  In  addi¬ 
tion,  as  will  be  seen  below,  the  Monte  Carlo  computation 
contains  implicit  assumptions  concerning  higher-order 
interactions  that  are  not  defined,  and  are  surprisingly 
difficult  to  state  precisely.  (To  say  they  are  implicit 
is  not  to  say  they  are  not  recognized  by  the  developers 
of  the  method,  only  that  the  nature  of  the  assumptions 
is  not  clearly  stated.) 

In  this  Report  an  elementary  model  of  probability 
cross-impacts  is  formulated  that  clarifies  the  notion  of 
"taking  account  of  interactions,"  and  does  this  without 
requiring  any  assumptions  concerning  higher-order  inter¬ 
actions.  The  model  is  based  on  fundamental  postulates  and 
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theorems  of  probability.  The  model  does  not  take  into 
account  all  of  the  aspects  of  an  interacting  system  that 
are  pertinent  —  in  particular,  it  neglects  time  as  a 
parameter.  More  generally,  it  neglects  nonprobabilistic 
order  effects. 

The  approach  is  elementary  in  two  respects.  First, 
the  type  of  probability  system  and  the  form  of  the  cross¬ 
impacts  (interdependence)  assumed  are  of  a  very  simple 
probabilistic  form.  Second,  the  notion  of  taking  account 
of  the  interactions  is  elementary.  It  can  be  described 
as  follows:  If  an  individual  or  a  group  estimates  a  set 
of  probabilities  of  events  and  this  set  contains  inter¬ 
active  terms,  then  the  set  may  be  inconsistent.  The 
purpose  of  computation,  then,  is  to  test  the  consistency 
of  the  set  of  estimates,  and  if  the  set  is  not  consistent, 
to  perform  the  smallest  reasonable  perturbation  on  the 
original  estimates  to  create  a  set  that  is  consistent. 

As  soon  as  the  consistent  set  is  achieved  —  from  this 
elementary  point  of  view  --  the  interactions  have  been 
"taken  into  account." 

It  might  not  hurt  to  amplify  this  point  slightly. 

If  we  assume  that  the  purpose  of  cross-impact  analysis  is 
to  arrive  at  the  best  possible  estimate  of  the  separate 
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probabilities  of  each  of  the  events,  then  regardless  of 
how  the  original  estimates  are  obtained,  they  should 
already  include  the  interactions  among  the  events.  The 
basic  assumption  of  cross-impact  analysis  is  that  the 
separate  probabilities  do  take  the  interactions  into 
account,  but  incompletely  so  that  some  modification  is 
needed. 

Another  assumption  is  that  the  cross-impact  estimates 
are  more  "solid"  than  the  absolute  probability  estimates. 
There  are  several  motivations  behind  this  assumption. 
First,  there  is  widespread,  and  probably  generally  sound, 
opinion  that  relative  probabilities  are  clearer  and  easier 
to  estimate  than  absolute  probabilities.  I  do  not  know 
of  any  experimental  data  to  support  this,  but  it  does 
appear  that  the  more  limited  reference  of  a  relative 
probability  makes  it  psychologically  easier  to  deal  with. 
Second,  there  is  an  argument  (which  may  or  may  not  have 
logical  justification)  that  narrowing  the  reference  class 


There  are  a  number  of  considerations  which  suggest 
that,  for  purposes  of  long-range  planning,  the  absolute 
probabilities  are  of  secondary  interest,  whereas  the 
"scenario"  probabilities,  i.e.,  the  probabilities  of  joint 
occurrence  or  nonoccurrence  of  many  events,  are  more 
directly  relevant.  This  topic  will  be  discussed  in  the 
text  in  greater  detail. 
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in  some  way  makes  the  probability  more  "correct."  This 
is  the  basis  for  Reichenbach’ s  recommendation  [Ref.  2, 
p.  374]  that  in  practical  applications  of  probabilities 
(decisions)  the  narrowest  reference  class  for  which  there 
is  reliable  information  should  be  used.  Finally,  and 
probably  most  important  for  cross-impact  analysis,  the 
notion  of  cross-impacts  is  new,  and  should  receive  greater 
emphasis. 

None  of  the  foregoing  justifies  the  assumption  that 
the  cross-impacts  are  more  solid  than  the  absolute  proba¬ 
bilities,  but  they  do  lend  some  heuristic  weight  to  the 
computational  structure.  These  considerations  suggest 
that  adjustments  should  be  made  in  the  absolute  probabil¬ 
ities,  not  the  cross- impacts.  It  will  be  shown  below  that 
this  point  of  view  cannot  be  maintained  strictly.  Is  is 
possible  that  inconsistencies  appear  in  the  cross- impacts 
as  well  as  in  the  original  estimates  of  absolute  probabil¬ 
ities.  However,  this  assumption  can  be  maintained  in  a 
weaker  sense  if  the  cross-impacts  can  be  adjusted  without 
making  use  of  the  absolute  probabilities,  and  then  the 
absolute  probabilities  can  be  adjusted  with  fixed  cross¬ 
impacts. 

The  results  to  be  presented  in  this  Report,  then, 
can  be  summed  up  by  saying  that  given  a  set  of  estimates  of 
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absolute  probabilities  and  cross-impacts  in  the  form  of 
relative  probabilities,  simple  tests  exist  for  determining 
the  consistency  of  the  cross-impact  matrix,  and  for  deter¬ 
mining  the  consistency  of  the  absolute  probabilities  given 
that  the  cross-impact  matrix  is  consistent.  If  the  set  of 
estimates  is  consistent,  then  no  further  computation  is 
required.  If  the  set  is  not  consistent,  then  a  number  of 
steps  may  be  taken  to  adjust  the  set,  ranging  from  simpli¬ 
fied  averaging  techniques  to  reiteration  of  the  estimates, 
given  a  display  of  the  inconsistencies  involved.  The 
adjustment  procedure  used  should  depend  on  the  nature  of 
the  inconsistencies  and  the  opportunities  for  querying  the 
estimators  again. 

The  consistency  condition  derived  below  takes  a  par¬ 
ticularly  elegant  form  when  the  cross-impact  matrix  is 
expressed  as  a  set  of  relative  probabilities,  that  is, 
the  probability  of  event  e^  is  p,  given  that  event  e. 
occurs.  With  cross-impacts  of  this  form,  the  consistency 
condition  can  be  stated  as  follows:  The  n  events  define 
an  n-dimensional  probability  space  (strictly  speaking  an 
n-dimensional  hypercube,  since  each  probability  can  vary 
only  between  0  and  1).  If  the  cross-impacts  are  mutually 
consistent,  they  define  a  single  line  in  this  hypercube, 
which  passes  through  the  origin.  A  set  of  absolute 
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probabilities  consistent  with  the  cross-impacts  will  then 
define  a  point  lying  on  this  line.  As  it  turns  out,  the 
condition  is  relatively  easy  to  test  and  allows  a  fairly 
simple  description  of  methods  of  resolving  inconsistencies 
if  the  estimates  do  not  pass  the  test. 
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II.  CONSISTENT  SYSTEMS  OF  PROBABILITIES 


We  will  be  concerned  with  an  elementary  system  of 
probabilities: 


A.  A  set  of  n  events  e. 


, »  • 


»  e^»  • • • »  en* 


B.  A  set  of  n  absolute  probabilities  of  these 

events,  Pfe^),  P(e2),  ...»  P(ei>,  ...»  P(en). 

2 

C.  A  set  of  n  -  n  relative  probabilities  of  the 
form  P(e^/e2),  ...»  P(e^/e^.),  ...  (read  as  "the 
probability  of  e^  given  that  e^  occurs"). 

D.  A  set  of  higher-order  probabilities,  illustrated 

by  P(e^  •  e  •  e^),  where  the  period  indicates 

joint  occurrence  (read  as  "the  probability  that 

e,,  ett  and  e,  all  occur), 
i  j*  k 


There  is  a  large  family  of  probabilities  of  type  D. 

Since  we  will  not  deal  formally  with  this  set,  they  are 

not  listed  in  detail. 

2 

The  set  of  n  -  n  relative  probabilities  of  type  C 
will  be  referred  to  as  the  cross -impact  matrix. 


The  notion  of  an  absolute  probability  is  sometimes 
misunderstood.  For  the  purpose  of  this  discussion, 
we  simply  assume  a  common  universe  of  discourse  for  the 
events,  and  refer  the  probabilities  to  that.  In  this 
respect,  the  absolute  probabilities  of  each  event  should 
reflect  (in  a  completely  buried  form)  all  of  the  inter¬ 
actions  between  that  event  and  all  others.  In  particular, 
the  absolute  probability  of  an  event  is  not  interpreted 
in  the  Bayesian  sense  of  an  a  priori  probability. 
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The  set  of  events  je^j  could  be  a  list  of  potential 
technological  developments,  a  list  of  social  or  political 
events,  a  combination  of  these,  or  something  entirely 
different,  such  as  a  set  of  symptoms  of  diseases  among 
the  total  U„S.  population. 

It  is  easy  to  show  that  for  a  complete  specification 

of  a  system,  2n  -  1  probabilities  must  be  given;  these 

are  independent  except  for  a  set  of  inequalities  illustrated 

by  P(e^  •  e.)  <  P(e_^).  In  general,  the  n  absolute  proba- 

2 

bilities  and  the  n  -  n  relative  probabilities  are  quite 
insufficient  to  completely  specify  the  system. 

Given  a  set  of  probabilities  of  the  forms  B  and  C, 
a  simple  question  can  be  asked;  namely,  are  they  a  consis¬ 
tent  set?  "Consistent"  here  means  compatible  with  the 
usual  calculus  of  probabilities.  The  question  is  meaning¬ 
ful  because  the  set  of  probabilities  in  the  forms  B  and  C 
is  redundant. 

There  are  two  kinds  of  redundancy.  The  first  involves 

2 

the  probabilities  of  type  C.  All  of  the  n  -  n  relative 
probabilities  in  the  cross-impact  matrix  can  be  replaced 
by  the  — ^  joint  probabilities  of  the  form  P(e^  •  e  ). 
(The  factor  1/2  comes  from  the  fact  that  joint  occurrence 
is  commutative.)  In  short,  there  are  twice  as  many  entries 
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in  the  cross-impact  matrix  as  are  needed  to  specify  all 
probabilities  involving  no  more  than  two  events. 

The  second  type  of  redundancy  involves  the  inter¬ 
relationship  of  the  absolute  and  relative  probabilities. 

Even  with  a  consistent  set  of  relative  probabilities,  the 
absolute  probabilities  may  not  combine  with  the  relative 
probabilities  in  accordance  with  the  rules  of  the  calculus 
of  probabilities. 

We  will  use  three  elementary  postulates  of  the  cal¬ 
culus  of  probabilities  and  one  theorem.  The  three  elemen¬ 
tary  postulates  are 

Pi:  Normalization.  0  <  p  <  1  for  any  probability  p. 

P2:  Rule  of  the  product.  PCe^  •  e.)  =  P(e^)  .  P(e./e ) 

J  J  ^ 

-  P(ej)  •  P(ei/ej). 

P3:  Rule  of  addition.  P(e.  or  e.)  «  P(e.)  +  P(e.)  - 

i  j  i  j 

p(el  •  ej). 

The  theorem  is  one  which  I  derived  in  my  Ph.D.  thesis, 
and  is  referred  to  in  Reichenbach  [Ref.  2,  p.  112].  It 
will  not  be  proved  here,  but  simply  stated. 

Theorem:  Rule  of  the  triangle. 

(1)  ?(et/ep  •  *  P^ej^ek^ 

=  P^eje^  •  P(ek/e^)  •  P^/e^ 
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The  theorem  states  that  for  a  set  of  three  events, 
the  product  of  the  relative  probabilities  multiplying 
around  the  triangle  in  one  direction  is  equal  to  the 
product  of  the  relative  probabilities  multiplying  in  the 
other  direction  (see  Fig.  1).  The  theorem  is  easily 
extended  to  four  or  more  events,  but  the  same  effect  can 
be  achieved  by  treating  the  larger  set  in  subsets  of  three. 

Since  we  can  assume  that  all  the  probabilities  given 
in  C  and  D  are  already  between  zero  and  one,  the  only  role 
of  PI  is  to  combine  with  P2  and  P3  to  give  the  weak  in¬ 
equality: 

(2)  P(ei)  +  P(ej)  -  P(ei)  .  P(ej/e1)  <  1 

P(ei)  +  P(ej)  -  P(ej)  .  PCej/ej)  <  1 


*t 


Fig.  1— Rule  of  the  triangle 
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The  rule  of  the  product  has  an  immediate  consequence: 


(3)  P(ej)  -  P^)  •  PCej/e^  /  P(fil/e  ) 


Eq.  (3)  can  be  interpreted  as  asserting  that  if  PCe^/e^)  and 
P(e^/e^)  are  fixed,  then  P(e^)  and  **(ej)  must  lie  on  a 
straight  line  through  the  origin  in  the  PCe^),  P(e^)  unit 
square,  as  illustrated  in  Fig.  2. 


Fig.  2— Geometric  representation  of  the 
rule  of  the  product 


/ 
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Similarly,  for  all  other  pairs  of  events,  the  rule 
of  the  product  implies  that  these  must  lie  on  a  straight 
line  through  the  origin  in  their  respective  unit  squares 
when  the  relative  probabilities  are  given.  Now,  if  we 
consider  the  space  of  all  the  absolute  probabilities- - 
the  unit  hypercube- -the  rule  of  the  product  in  conjunction 
with  the  rule  of  the  triangle  assures  that  all  of  the 
absolute  probabilities  must  lie  on  a  single  straight  line 
within  the  hypercube.  To  illustrate  this  theorem,  we 
first  consider  the  unit  cube  defined  by  three  events. 

Eq.  (3)  asserts  that  the  probabilities  must  lie  on  the 
intersection  of  the  planes  defined  by  the  straight  lines 
in  the  respective  unit  squares. 

Figure  3  illustrates  the  intersection  of  the  two  planes 
defined  by  the  pairs  P(e^),  P(e^)  an(*  P(ej),  There 

is  one  additional  plane  defined  by  the  pair  P(e^),  P(ej); 
and  the  intersection  of  it  with  the  first  two  produce  two 
additional  lines.  If  the  relative  probabilities  are  con¬ 
sistent  (by  the  rule  of  the  triangle  and  the  rule  of  the 
product)  the  three  lines  will  coincide.  Figure  3  portrays 
an  inconsistent  case. 

To  present  the  general  consistency  condition  for  cross¬ 
impact  matrices,  it  is  convenient  first  to  establish  a  lemma 


onsistent  relative  probabilities 
:or  three  events 


<1 
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concern  ing  lines  in  n-dimensional  Euclidean  spaces.  A 

line  is  defined  by  two  points  X  ■  (x^»  3^ . xn)  anc* 

Y  ■  (y, ,  y. . .  ).  Any  other  point  Z  on  the  line 

is  a  linear  combination  of  these  two;  i.e. ,  Z  ■  aX  +  (1  -  a)Y, 
-  •  <  a  <  «».  It  is  convenient  to  shift  the  origin  to  Y, 
in  which  case  Z  -  Y  ■  a(X  -  Y).  Renaming  Z  -  Y  -  Z*  and 
X  -  Y  -  X',  we  have  Z*  -  aX'. 

Lemma  1:  A  necessary  and  sufficient  condition  that 
a  matrix  S  »  where  s^  **  l/s^ ,  s^  =  1,*  define 

a  line  aX  through  the  origin  in  n-dimensional  Euclidean 
space,  such  that  the  slope  x^/x^  the  line  projected  on 
the  two-dimensional  subspace  (i,j)  is  s^,  is  the  triangle 


rule 


Sij  *  Sjk  *  Ski 


Proof: 


Necessity.  Assume  there  is  a  line  aX  with  the 
hypothesized  properties,  then 


xi  xi  \ 


1  «*  s 


ij  *  sjk  *  Ski 


Sufficiency.  Consider  a  matrix  S  that  fulfills  the 

i 

triangle  rule.  Define  a  point  X  by 


To  eliminate  inessential  special  cases,  s,  .  is  also 
assumed  to  be  nonzero.  ^ 
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The  triangle  rule  of  Eq.  (a)  implies  that 


s 


lj  Ji 


where 

and  since 
(b) 


il 


s . . 

Ji 


’il 


’ji 


su  •  sn 


=  s 


li 


Ill 

slj  xj 


ij 


P(e  /e1)/P(ei/ej).  The 


Since  for  any  other  point  Y  on  the  line  ax,  y^/y 
0x^/3x^  *=  x^/xy  Eq.  (b)  is  completely  general. 

Lemma  1  is  applicable  to  a  cross-impact  matrix  by 
defining  a  matrix  S:  s 
conditions  s^  ■  1/s  Sii  *  ^  ^°^ow  Immediately  from 

the  definition,  and  the  triangle  rule  Eq.  (a)  follows 
from  the  rule  of  the  triangle  for  relative  probabilities. 

The  rule  of  addition  can  be  invoked  by  determining 
the  intersection  of  the  lines  defined  by  setting  the 
inequalities  to  equalities  in  Eq.  (2).  Thus, 

(4)  P(et)  +  P(ej)  -  P(ei)  •  P(e^/e±)  «  1 

P(ei)  +  P^)  -  P(ej)  •  PCe^ej)  -  1 


Solving  these  two  for  P(et>  and  P(e^)  gives 


-18- 


It  is  simple  to  verify  that  the  pair  P^),  P^)  lies 

on  the  line  defined  by  Lemma  1. 

The  question  still  remains  whether  the  limits  imposed 

by  different  applications  of  Eq.  (5)  with  different  pairs 
result  in  the  same  limit.  Thus,  for  example,  we  have 


P(ei/ek) 

P(£i)  *  P(e,/eJ  +  P(e./e.)  -  PU,/e  )  •  P<VV 


In  general,  this  is  not  the  same  limit  as  expressed  by 
Eq.  (5)  above,  thus  the  minimum  of  all  the  limits  deter¬ 
mined  by  all  pairs  must  be  selected.  The  minimum  fixes 
a  point  L  on  the  consistency  line  in  the  hypercube;  any 
acceptable  point  is  lower  than  or  equal  to  that  point. 

This  completes  the  set  of  consistency  conditions. 

In  sum,  the  consistency  conditions  define  a  line  passing 
through  the  origin  in  the  unit  hypercube  and  a  point  on 
that  line.  To  be  consistent,  the  absolute  probabilities 
must  lie  on  the  segment  of  that  line  between  the  origin 
and  the  given  point. 
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The  foregoing  demonstrates  the  necessity,  but  not  the 
sufficiency,  of  the  conditions.  A  direct  consequence  of 
the  rule  of  addition  (P3)  is  that  the  probability  of  the 
disjunction  of  any  subset  of  the  absolute  probabilities 
must  be  less  than  or  equal  to  one.  The  disjunctive  pro¬ 
babilities  for  subsets  larger  than  two  cannot  be  computed 
from  the  absolute  probabilities  and  binary  relative  prob¬ 
abilities  since  the  disjunctive  probabilities  of  sets 
larger  than  two  involve  higher-order  interactions. 


/ 


III.  RESOLUTION  OF  INCONSISTENCIES 


In  previous  sections  we  defined  a  set  of  conditions 
to  be  met  for  a  set  of  absolute  probabilities  and  a  cross¬ 
impact  matrix  of  relative  probabilities  to  be  consistent. 
The  simplest  application  of  the  conditions  for  the  matrix 
is  to  test  all  triplets  of  relative  probabilities  by  the 
triangle  rule.  This  test  is  tedious.  There  are 
n (n..  -I- --1  triplets  for  n  events.  For  50  events 

there  are  19,600  triplets. 

However,  it  is  necessary  to  check  only  a  subset  of 
the  triplets,  a  convenient  subset  being  the  set  of  tri¬ 
angles  having  one  event  in  common.  The  number  of  triangles 
in  such  a  subset  is  Thus,  the  number  of 

independent  triangles  increases  as  the  square  of  n,  rather 
than  as  the  cube  which  is  the  case  for  the  total  number  of 
triangles.  This  happy  situation  is  guaranteed  by  the  fol¬ 
lowing  lemma.* 

Lemma  2;  If  the  rule  of  the  triangle  holds  for  all 
triangles  that  have  one  event  in  common,  then  it  holds  for 
all  triangles. 

*1  would  like  to  thank  T.  Brown  and  J.  Spencer  for 
helpful  suggestions  concerning  this  simplification  of  the 
consistency  test. 


srq&np. 
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Proof:  Consider  any  event  e^,  and  any  triangle  e^, 
e j ,  e^,  not  including  e^.  Assume  that  the  rule  of  the  tri¬ 
angle  holds  for  all  triangles  containing  e^.  Then,  abbre¬ 
viating  P(e^/ej)  to  P(i/j)  gives 

P«7i)  •  P(i/j)  •  P(J/0  -  P(i/0  •  P(j/i)  •  P«7j) 

P(kM)  •  P(j/k)  •  P(*,/j)  -  P^/k)  •  P(k/j)  •  P(j/*,) 

P(i/t)  •  P(k/i)  •  P(-t/k)  -  P(-t/i)  •  P(i/k)  •  P(kM). 

Multiplying  the  three  expressions  on  the  left  side 
of  the  equations  and  the  three  expressions  on  the  right 
side,  the  terms  containing  l  cancel  and  we  arrive  at 

P(i/j)  •  P(j/k)  •  P(k/i)  -  P(j/i)  •  P(k/j)  •  P(i/k) 

which  was  to  be  proved. 

For  the  absolute  probabilities,  assuming  the  matrix 
is  consistent,  the  simplest  test  is  first  to  calculate  the 
limits  imposed  by  the  rule  of  addition.  Assuming  that  test 
is  passed,  the  individual  probabilities  can  be  tested  by 

starting  with  P(e^)  and  computing  the  others  by  the  rule 

PCe^  -  Pfep  •  PCe^ep 


It  appears  likely  that  in  most  practical  applications 
the  sets  of  probabilities  will  not  be  consistent,  and  a 
question  arises  as  to  the  method  of  proceeding.  There  is 
no  "correct"  method  of  resolving  the  inconsistencies.  There 
are  several  directions  that  can  be  taken,  depending  on  the 
interest  of  the  study  manager,  on  the  availability  of  re¬ 
spondents  for  reestimation,  and  the  like. 

Assuming  no  restrictions  on  reestimation,  it  appears 
desirable  to  present  the  information  concerning  inconsis¬ 
tencies  to  the  respondents,  and  obtain  reestimates  from 
them.  This  poses  the  question  of  how  the  information  is 
to  be  presented  to  be  useful  to  the  respondents. 

One  reasonable  display  can  be  obtained  by  computing 
the  adjustments  described  below,  and  feeding  back  the  orig¬ 
inal  and  adjusted  estimates. 

Assuming  that  the  cross-impact  matrix  is  consistent, 
but  the  set  of  absolute  probabilities  does  not  lie  on  the 
line,  a  natural  and  simple  adjustment  is  to  choose  the  point 
on  the  line  that  is  nearest  to  the  estimated  point,  as  illus¬ 
trated  for  two  dimensions  in  Fig.  4.  The  computation  of  the 
nearest  point  is  simple,  given  the  line  in  parametric  form. 

If  P  is  the  estimated  point  and  aX  is  the  line,  then  the 
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desired  nearest  point  on  the  line  Z  is  given  by 


E  p.x. 

(6)  Z  =  px,  p  - 


Fig. 4— Adjustment  of  absolute  probabilities  by  computing  the 
nearest  point  on  the  line  defined  by  the  cross-impact  matrix 

If  Z  lies  above  the  limit  L  imposed  by  the  rule  of 
addition  (Z'  in  Fig.  4),  the  most  natural  rule  is  to  select 
L  as  the  adjusted  set  of  probabilities. 

The  situation  is  not  quite  as  neat  if  the  cross -impact 
matrix  itself  is  inconsistent.  The  problem  here  is  that 
inconsistencies  result  in  a  set  of  lines,  which  can  become 
very  large  rapidly,  and  there  is  no  obvious  way  of  weighting 
these  in  the  computation  of  a  representative  line.  A  rela¬ 
tively  simple  adjustment,  and  one  that  appears  sufficient 


/ 
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« 


for  the  purpose--especially  if  the  group  is  expected  to 
reestimate — is  the  following: 


(7) 


x,  -  -  E. 
i  n  j 


P(ei/el> 

h  PW: 


This  computation  arises  from  allowing  each  row  in  the  matrix 
in  turn  to  generate  a  potential  line,  and  averaging  these 
lines  by  summing  their  intersection  with  the  unit  sphere. 
Although  Gq.  (7)  does  not  arise  from  any  optimization  rule, 
it  does  assure  that  in  adjusting  X  the  contributions  of  all 
other  events  to  e^  will  be  "taken  into  account,"  and  the 
contribution  of  e^  to  all  the  other  events  enters  in  the 
normalization  to  the  unit  sphere. 

A  more  satisfying  variant  of  the  procedure  in  Eq.  (6) 
would  be  to  make  use  of  group  weights  on  the  probabilities. 
That  is,  each  member  of  the  group  can  be  requested  to  eval¬ 
uate  how  confident  he  is  of  his  estimate  (with  some  suit¬ 
able  rating  scale)  and  a  group  measure  of  confidence--e.g. , 
the  average  of  the  individual  ratings — can  be  used  to  weight 
the  absolute  probability  estimates. 

If  the  weights  w^  are  normalized  so  that  0  <  w^  <  1, 
where  1  indicates  certainty  and  0  indicates  sheer  guess, 
then  a  reasonable  weighted  form  of  Eq.  (6)  would  be 


1 


A  weighted  adjustment  for  the  relative  probabilities 
is  much  more  complex;  a  reasonable  form  has  not  yet  been 
derived.  One  of  the  difficulties  is  that  the  consistency 
condition  furnished  by  the  rule  of  the  triangle  deals  only 
with  the  ratios  of  the  relative  probabilities  and  not  with 
the  probabilities  themselves.  Ratios  of  estimated  weights 
are  somewhat  awkward  to  use  here. 
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IV.  SCENARIOS 

The  cross-impact  computation  developed  at  The  Insti¬ 
tute  for  the  Future  not  only  produces  a  Monte  Carlo  esti¬ 
mate  of  revised  absolute  probabilities,  but  also  gives  a 
Monte  Carlo  estimate  of  the  probabilities  of  joint  occur¬ 
rence  of  the  total  set  of  events.  Each  chain  of  events 
is  a  sample  out  of  a  large  population  of  potential  chains. 
The  number  of  potential  chains  is  n! 2n.  The  factor  2n 
arises  from  the  total  number  of  joint  occurrences  (posi- 
tive  or  negative)  of  n  events,  and  the  factor  n!  from  the 
manner  in  which  the  next  event  in  the  chain  is  selected, 
namely,  by  considering  each  remaining  event  equally  likely 
to  be  next. 

The  computation  involves  a  strong  independence  assump¬ 
tion,  namely,  that  the  change  in  the  likelihood  of  occur¬ 
rence  of  a  given  event  is  influenced  only  by  the  preceeding 
event  that  has  occurred.  (In  some  forms  of  the  computation 
the  change  is  assessed  in  terms  of  occurrence  or  nonoccur¬ 
rence  of  the  preceeding  event.)  In  general  probability 
systems,  this  assumption  is  not  only  not  fulfilled;  the 
assumption  has  as  a  consequence  that  the  system  is  degen¬ 
erate.  A  simple  example  may  suffice  to  show  this.  If  we 
make  an  assumption  that  all  higher-order  interactions  are 


.  I  - 
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determined  solely  by  the  first-order  interactions,  then 
we  would  have 


(9)  P(ek/ei  *  ej)  =  P(ek/ei^  “  P(ek/ej^ 

Applying  the  assumption  to  all  interactions  would  imply 
that  all  the  cross -impacts  for  a  given  target  event  are 
equal.  The  same  result  follows  from  other  forms  of  the 
same  assumption,  for  example, 


(10)  P(ek  •  e^/e^  -  P(ek  •  e^) 


(ID 


P(ei  *  ei  *  ek^  P(ek) 

p<ei  •  ej  •  V  '  ^ 


where  the  bar  over  an  event  indicates  negation. 

The  Monte  Carlo  computation  developed  by  Gorden  el:  al. 
for  the  likelihood  method  involves  a  slightly  weaker  assump- 

*  t  ,  ,  I 

tion  than  Eq.  (9).  However,  it  is  more  complicated,  and 

I  *  J 

rather  than  try  to  deal  with  it  analytically,  a  simple  il¬ 
lustration  will  indicate  thp  effect  of  the  assumption. 

I  1  1  ,  l 

Consider  the  following  cross-impact  matrik: 

i  i 

.  1 

i  :  i 

•  '  1  .  i  1 

I 

‘  ,  ,  1 

•  I  ! 

1  <  .  i  i  : 

*'  l  ! 


I 


I 
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P(ei)  -  P(e2)  -  P(e3)  =  0.25 


Let 


1  -  P(e .)  P(ei/ei) 

"1J "  p(ej>  '  1  •  p(ej/ei> 

that  is,  is  the  factor  by  which  the  odds  for  e^  are 
multiplied  to  generate  the  odds  for  e ^ ,  given  that  e^  has 
occurred.  Then 

ha  -  HHSfi  - 3  -  *u  foralli  'J 

Each  event  has  a  positive  impact  on  the  others. 

If  the  Monte  Carlo  computation  is  carried  through, 
PCe^)  is  adjusted  from  0.25  to  0.32.  The  computation  sug 
gests  that  in  terms  of  the  positive  impacts  the  absolute 
probabilities  should  be  higher.  However,  the  original 
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probabilities  are  fully  consistent  with  the  cross-impacts; 
the  increase  is,  in  this  sense,  unwarranted.* 

For  purposes  of  evaluating  policies,  the  probabilities 
of  the  chains  (where  each  chain  can  be  considered  one  sce¬ 
nario)  are  of  greater  interest  than  the  absolute  probabil¬ 
ities  of  the  separate  events.  We  have  indicated  that  to 
make  the  scenario  computation  useful  it  will  be  necessary 
either  to  obtain  estimates  of  the  higher-order  interactions 
or  find  a  more  logically  correct  assumption  concerning  them. 
The  first  alternative  is  rather  discouraging.  The  number 
of  higher-order  probabilities  becomes  somewhat  astronomical 
with  large  sets  of  events.  The  second  alternative — although 
more  attractive  in  terms  of  the  feasibility  of  obtaining 

estimates--is  subject  to  the  danger  of  artificiality. 

*  *  * 

This  Report  presents  an  elementary  approach  to  the 
estimation  of  complex  probability  systems.  Above  all,  it 
contains  no  consideration  of  the  structural  properties  of 
the  subject  matter  involved.  Generally  speaking,  it  can 
be  expected  that  a  given  application,  whether  it  is  tech¬ 
nological  events  or  health  categories,  will  have  certain 

*Although  I  have  not  made  the  calculation,  it  appears 
likely  that  most  of  the  adjustment  in  the  cases  presented 
in  Ref.  [1]  are  of  this  sort. 
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underlying  characteristics  that  will  affect  both  the  binary 
cross -impact  probabilities  and  the  higher-order  interactions 
(e.g.,  with  the  health  categories,  it  is  very  unlikely  that 
any  individual  will  simultaneously  be  afflicted  with  a  large 
number  of  pathological  conditions).  Convenient  methods  of 
expressing  these  structural  properties  have  not,  to  my  know¬ 
ledge,  been  defined  for  the  kinds  of  systems  in  which  cross¬ 
impact  analysis  would  appear  to  be  applicable.  My  first 
impression  is  that  such  structural  properties  may  furnish 
the  required  assumption  concerning  higher-order  probabilities. 

The  resolution  of  inconsistencies  in  the  cross-impact 
matrix  expressed  in  Eq.  (7)  can  probably  be  developed  further 
and  appears  to  be  an  interesting  mathematical  problem  in 
its  own  right. 
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